System and method for managing complex relationships over distributed heterogeneous data sources

ABSTRACT

A relationship and navigation data repository can interface with a central data model and contains templates that define relationships among data from a number of distributed heterogeneous data sources. An integration engine is coupled to the relationship and navigation data repository, and can receive a query command and determine which of the heterogeneous data sources and which of the templates the query applies to, and then calculate the desired query result, responsive to the command, based on the relevant data sources and templates. The distributed heterogeneous data sources are managed by a system and method that involves obtaining pre-existing definitional data, instantiating in-memory nodes for the data, initializing a relationship attribute and an entity attribute for each of the nodes, and then forming an updated navigation tree structure therefrom.

FIELD OF THE INVENTION

The present invention relates to the data processing arts, and moreparticularly relates to a system and method for managing relationshipsamong a plurality of distributed heterogeneous data sources.

BACKGROUND OF THE INVENTION

Integration of data from a variety of data sources is desirable in manyenterprises. At present, attention has been focused on straightforwarddata querying and mapping in a distributed environment of heterogeneousdata sources, involving data base management systems, knowledge bases,flat file systems, forms and spreadsheet applications, and the like.Such data querying typically allows users to specify search criteriausing a query language such as structured query language (SQL), whiledata mapping is used to translate the queries and results between theheterogeneous back-end systems and the centralized integration platform.Although large-scale integration solutions are currently being built tooffer efficient data retrieval, none support an infrastructure forcomplex relationship management of the data. Currently, specificationsfor data integration typically involve only explicit data mapping frommultiple data sources. For example, U.S. Pat. No. 6,633,889 to Desslochet al. teaches the mapping of persistent data objects residing inmultiple data sources into a single, reusable software componentaccessible to an object-oriented programming language applicationperformed by a computer, for multi-data base access to data that may bephysically distributed and stored in disparate database managementsystems. A single virtual data object can be created based on registereddata objects, and the virtual data object may be wrapped as a reusablesoftware component.

Prior art solutions typically do not promote the use and identificationof implicit information, i.e., complex relationship definition, on topof traditional explicit mapping. Further, in prior art systems, the usermust traditionally issue multiple queries to piece together a finalanswer, for example, to develop a management chain. The user typicallymust have an overall view of the system's schema and structure inadvance in order even to formulate appropriate queries. Finally, priorart systems are typically limited to the retrieval of explicit data anddo not allow retrieval and maintenance of logical implications betweenthe explicit pieces of data through generalized data structures.

In view of the foregoing, it is desirable to provide a system and methodfor managing complex relationships over distributed heterogeneous datasources. The system and method should promote use and identification ofimplicit information, that is, complex relationship definition, inaddition to traditional explicit mapping, and should provide the userwith the flexibility of defining any sort of desired relationship in aconceptually intuitive and visually explicit manner using, for example,extensible markup language (XML) scripts. It would also be desirable ifthe system and method permitted easy navigation by the user withoutrequiring complex query formulation. Yet further, it would be desirableif the system and method allowed users to retrieve additional “implicit”data by treating the navigation structure and assembly itself as being acomplex form of data.

SUMMARY OF THE INVENTION

The present invention, which addresses the needs identified in the priorart, provides a system and method for managing complex relationshipsover distributed heterogeneous data sources. In one aspect, a systemintegration platform is provided which manages the plurality ofdistributed heterogeneous data sources as well as a central data modelthat is configured to contain entities and/or attributes that are mappedand/or translated from the data sources. The management can be inresponse to a query command, for example, from a user. The platform candetermine a desired query result corresponding to the command. Theplatform can include a relationship and navigation data repository thatis configured to contain relationship templates that definerelationships among the data from the different data sources, and alsoto interface with the central data model. The system integrationplatform can further include an integration engine coupled to therelationship and navigation data repository. The engine can beconfigured to receive the query command, determine which of the datasources and which of the templates the query command applies to, and tocalculate the desired query result responsive to the query command.

In another aspect of the invention, a method of constructing anavigation tree structure for managing a plurality of distributedheterogeneous data sources is provided. The method include the steps ofobtaining pre-existing definitional data, instantiating a plurality ofin-memory nodes for the pre-existing definitional data, initializing atleast a relationship attribute and an entity attribute for each of thenodes, and forming an updated navigation tree structure from the givennodes having the given attributes.

In still a further aspect of the present invention, a method ofprocessing a query command associated with a plurality of distributedheterogeneous data sources is provided. The method includes obtainingthe query command, and, responsive to the command, either creating anavigation session or loading a pre-existing navigation session. Themethod further includes passing the query command and either the newnavigation session or pre-existing navigation session to an integrationengine, and then identifying and interpreting, by using the integrationengine, pertinent definition and computation templates that areassociated with the query command. The method further includes computingand retrieving a desired query result, based at least in part on atleast one of the definition and computation templates, from pertinentdistributed heterogeneous data sources.

In yet a further aspect of the present invention, a system forconstructing a navigation tree structure for managing a plurality ofdistributed heterogeneous data sources is provided. The system includesa relationship and navigation data repository and an integration engine.The integration engine is coupled to the relationship and navigationdata repository, and is configured to obtain pre-existing definitionaldata from the relationship and navigational data repository and/or oneor more of the distributed data sources. The integration engine isfurther configured to instantiate a plurality of in-memory nodes for thedefinitional data and to initialize at least a relationship attributeand an entity attribute for each of the nodes. The integration engine isfurther configured to form an updated navigation tree structure from thegiven nodes having the given attributes.

A more complete understanding of the present invention, as well asfurther features and advantages of the present invention, will beobtained by reference to the following detailed description anddrawings.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 shows a systems integration platform in accordance with thepresent invention;

FIG. 2 shows a flow chart for a method of processing a query command inaccordance with the present invention;

FIG. 3 shows exemplary relationship definition and computation templatesin accordance with the present invention;

FIG. 4 shows a navigation tree structure configured for display to auser;

FIG. 5 shows a navigation tree and a node data array of a navigationtree structure in accordance with the present invention;

FIG. 6 shows a generalized navigation tree structure;

FIG. 7 shows a multi-dimensional navigation tree structure; and

FIG. 8 shows a method of constructing a navigation tree structure inaccordance with the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Reference should now be had to FIG. 1 which depicts a system integrationapparatus (also referred to as a platform) 100 according to anembodiment of the present invention. The platform 100 manages aplurality of distributed heterogeneous data sources 102 and a centraldata model 104. The central data model 104 is configured to containentities and/or attributes that are mapped and/or translated from thedata sources 102. The platform 100 determines a desired query result inresponse to a query command, which can be received, for example, from anexternal browser 106. It will be appreciated that (just as in atraditional integration platform using data federation) each of the datasources 102 can have data that can flow into the platform 100 through,for example, appropriate connections and wrappers (such as a JDBC (Javadatabase connectivity) wrapper or a custom API (application programinterface) wrapper).

The platform 100 includes a relationship and navigation data repository108 that is configured to contain relationship templates 110 that definerelationships among the data from the data sources 102. The relationshiptemplates 110 define relationships among data within individual datasources 102, and also define relationships among data in different datasources 102. The relationship and navigation data repository 108 isconfigured to interface with the central data model 104. Therelationship and navigation data repository 108 can optionally include anavigation assembly and structure repository 112, and the central datamodel 104 can optionally include entity definitions 114 andmapping/translation templates 116.

The system integration platform also includes an integration engine 118that is coupled to the relationship and navigation data repository 108.The integration engine 118 is configured to receive the query command,and to determine which of the plurality of data sources 102 and which ofthe templates contained in block 110 the query command applies to, so asto identify pertinent data sources and pertinent templates. Theintegration engine is further configured to calculate the desired queryresult in response to the query command, based on the pertinent datasources and the pertinent templates. The integration engine 118 can beimplemented as, for example, an XML engine that determines which datasources 102 a query applies to and the methods and parameters forcomputing the relationship in each pertinent back end system 102. Thelatter function can be performed, for example, by interpretingrelationship definition and computation templates 110 (for example, inthe form of scripts) in repository 108. Results can be mapped/translatedto the central data model 104 if desired.

The integration engine 118 can optionally be part of an applicationserver 120. The application server can also include an interface module122 that is coupled to the integration engine 118 and through which theintegration engine 118 receives the query command. By way of example, asshown in FIG. 1, the interface module can preferably be constructedusing an appropriate servlet/JSP (Java server pages) container 124 andappropriate JAVA beans and classes 126. However, any type of appropriateapplication (C++ or Visual Basic, for example) working through a browserinterface can be employed instead. The integration engine can optionallycalculate the result formed as a navigation tree structure (to bediscussed more fully below) that has a plurality of source data items,and the interface module 122 can be configured to format at least partof the navigation tree structure in a manner adapted to be output fromthe platform 100 in a readily accessible fashion (for example, to theexternal browser 106). Once the integration engine 110 has determineddesired results, the results, together with at least desired portions(such as active portions) of the navigation tree data structure can bepassed back to servlets 124 in interface 114 for display of the tree andupdating of the interface, and/or can be persisted, e.g., to repository112. The interface module 122 can optionally format at least part of thenavigation tree structure into a query interface 400, to be discussedbelow with respect to FIG. 4. The relationship and navigation datarepository 108 can be configured to store the navigation tree structurein block 112.

Continued reference should be given to FIG. 1 for discussion of a systemfor constructing a navigation tree structure for managing a plurality ofdistributed heterogeneous data sources 102. The elements of the systemfor constructing the navigation tree structure include the relationshipand navigation data repository 108 and the integration engine 118, whichis coupled to the relationship and navigation data repository 108. Theintegration engine 118 is configured to obtain pre-existing definitionaldata from the relationship and navigation data repository 108 and/or atleast one of the distributed data sources 102, and to instantiate aplurality of in-memory nodes for the definitional data. Further, theintegration engine is configured to initialize at least a relationshipattribute and an entity attribute for each of the nodes, such that aplurality of relationship attributes are initialized. At least some ofthe relationship attributes define relationships among data in differentones of the data sources. The relationship attributes and entityattributes of the nodes can correspond, for example, to any of items502, 510, 512, 514, 516 discussed below with respect to FIG. 5. Finally,the integration engine 118 can be configured to form an updatednavigation tree structure, for example, of the type described herein,from the given nodes having the given attributes. Thus, it will beappreciated that the system elements just discussed are one possiblesystem that can implement the method discussed below with respect toFIG. 8. Accordingly, the pre-existing definitional data can be a resultstored in one of the data sources 102, a result stored in therelationship and navigation data repository 108, or a sub-graph storedin the relationship and navigation data repository 108. The integrationengine 118 can then be configured to carry out any of the additionalmethod steps described below with respect to FIG. 8.

Reference should now be had to FIG. 2 which depicts a flow chart 200 ofa method, in accordance with the present invention, of processing aquery command associated with a plurality of distributed heterogeneousdata sources such as data sources 102. The method includes the steps ofobtaining the query command, as in block 202, and then, responsive tothe query command, either creating a navigation session or loading apre-existing navigation session, as per block 204. The command can beobtained, for example, from external browser 106 through interface 400(discussed below). The method includes the further step of passing thequery command and either the new navigation session or the pre-existingnavigation session that has been loaded to an integration engine, suchas integration engine 118, as shown in block 206. The method furtherincludes the step of identifying and interpreting pertinent definitionand computation templates associated with the query command, as shown atblock 208. The identifying and interpreting step is typically performedwith the aid of the integration engine 118, and at least some of thetemplates define relationships among data in different ones of the datasources 102. Finally, at block 210, the step of computing and retrievinga desired query result is depicted. The desired query result willtypically be based at least in part on at least one of the definitionand computation templates 110, and the desired query result is typicallyretrieved from pertinent ones of the distributed heterogeneous datasources 102 using the relationship information from the definition andcomputation templates. The desired query result can be assembled into anavigation tree structure using techniques as described above, and adesired portion can be formatted for output, as described above, while aportion not desired for output can be persisted to memory. These stepsare depicted in blocks 212, 214, 216. (Refer also to examples of activeand inactive regions given below with respect to FIG. 7.) Accordingly,the desired query result can be written to persistent storage formattedas a navigation tree structure or can be formatted for output andsubsequent display, and portions of the navigation tree structure can beselected for either the storage or display options, as desired.

Attention should now be given to FIG. 3 which depicts exemplary contentsof the relationship and navigation data repository 108, which wouldtypically be stored in the relationship definition and computationtemplates 110. Appropriate scripts and/or classes can be employed;templates for defining new relationships and XML scripts for existingrelationship definitions can be used as desired. Exemplary XMLdefinition and computation scripts, as well as associated classes, arepresented in FIG. 3 for the relationship “subassembly-parts.” Thedefinition script 300 can identify the source entity 302 (that is, theentity type of the source item) as well as a destination list 304including pertinent back-end data sources relevant to the relationship.In the example shown in FIG. 3, the source entity is of the type PART,such as, e.g., an automobile engine, which can be navigated by itssubassembly, and at least two back-end sources are specified(corresponding, for example, to two of the distributed data sources 102in FIG. 1). That is to say, at least two of the back-end sources containinformation that should be included in the subassembly of the sourceentity, for example, the fan blades for the automobile engine. Withinmethod tags 306, the exemplary script 300 also specifies computationspecific scripts 308, 310 which describe how to compute the desiredrelationship. The computation specific scripts can include a variety ofitems, including JDBC queries, web services invocations, compiled proxycode, proprietary application program interfaces (APIs), and the like.The actual classes can be as long as desired, depending on thecomplexity of the relationships to be defined. When the computationmechanism involves compiled class files 312, the specification scriptcan include the parameters and outputs of each function in the correctcalling order, where the input parameters are matched to the sourceentity's attributes or previous output values.

While a specific example has been shown in FIG. 3, it will beappreciated that, in general, the relationship and navigation datarepository 108 of platform 100 can include a plurality of relationshipdefinitions defining a plurality of relationships; each of thedefinitions can in turn identify a source entity 302, an associateddestination list 304 of pertinent ones of the data sources, and at leastone computation definition reference 306 to appropriate definitions 308,310 and the like. The computation definitions such as 308, 310 candefine how to compute corresponding ones of the relationships, inresponse to being referenced by a computation definition reference 306.

Referring now to FIG. 4, the interface module 122 can optionally formatat least part of the navigation tree structure into a query interface400. Interface 400 can include data representative of a first portion402 for specification of a desired one of the source data items from thenavigation tree structure and can also include data representative of asecond portion 404 for specification of a pertinent one of therelationship templates. If desired, the query interface 400 can includedata representative of a third portion 406 that specifies a desireddepth of navigation into the navigation tree structure. The portion 402can display the user's navigation thus far in a given session. Theportion 408 can display the results of a current query.

In preferred embodiments of the invention, querying can take place witha minimal number of selections or inputs, for example, by mouse clicks.For example, a user can click on a source data item 410 and a desiredrelationship 404 by which to query. The example shown in FIG. 4 depictsa subassembly parts breakdown for an exemplary “Acme X12” automobile. Asnoted, where desired, the query interface can include data that isrepresentative of portion 406 for specifying the depth of navigationinto the navigation tree structure. The “depth” option allows the userto specify how many levels (an integer number) to traverse, while the“all” option traverses all levels. Typically, both options can expandthe navigation by recursively applying the query to all nodes or resultsin the results set in a depth-first manner. For example, clicking on“John Smith,” “manager,” and “all” will retrieve the entire managementchain starting with John Smith and up to the Chief Executive Officer.Clicking on “Acme X12,” “subassembly-parts,” and “depth” specifying adepth of 3, will retrieve a tree hierarchy of all the sub-components ofthe automobile to a depth of 3. Thus, in the exemplary embodimentdepicted in FIG. 4, operations for navigating through the data andpersisting the data (using the save option in portion 406) and forspecifying further querying options can be simply triggered throughclicks in the depicted interface. Interface 400 can preferably beimplemented through Java servlets and Java server pages (JSP pages)tied, e.g., to Java beans and classes (see items 124, 126 in FIG. 1). Asnoted above, other appropriate applications such as C++ and Visual Basiccan be employed. Interface 400 can manage the relationship andnavigation structure for the session in conjunction with integrationengine 118, and can reside on application server 120, or all or part ofit can be downloaded to a user machine running browser 106 (elements106, 118, 120 have been discussed above with respect to FIG. 1).

Now viewing FIG. 5, aspects of the navigation tree structure 500 of thepresent invention are depicted. As noted above, the navigation treestructure 500 can have a plurality of source data items; the source dataitems can typically include the first item specified, for example byclicking, at any level. Source data items are depicted in portion 402shown in FIG. 4, and corresponding display attributes 502 are shown inFIG. 5. The navigation tree structure 500 can comprise a navigation treeportion 504 and a node data array 506. The navigation tree portion 504is essentially a hierarchy of lists; as shown in FIG. 5 it representsthe top levels depicted in FIG. 4. The navigation tree structure 500 andportion 504 are designed to aid in constructing the interface of FIG. 4,and the hierarchy of lists allows for simple building and formatting ofthe navigation information into the interface seen by users; forexample, in the form of a web interface. The representation seen in FIG.4 is a natural representation of a depth-first traversal of thenavigation tree structure. “Depth-first traversal” is a term of artknown to those skilled in graph theory; one proceeds down as far aspossible to a leaf node and then moves to the next sibling. It is to becontrasted to a “breadth-first” traversal where one proceeds through allsiblings first. The navigation tree portion 504 can be employed forpresentation purposes, and the nodes 508 in the tree can be configuredto hold data for presentation. Each node 408 can include theaforementioned display attributes 502, to identify each data item in thenavigation tree structure in the interface. The relationship portion 510can store the relationship that is used to query with, in the event thatthe data item served as the source entity in a query. The next levelportion 512 can maintain a list of data items in the next immediatelevel down of navigation. Finally, the node number 514 can be used as anindex for handling requests, for example, in dynamic web pages.

The navigation tree structure 500, as noted, can also include a nodedata array 506. Appropriate attributes 516 associated with a given nodecan then be used as parameters, for example, to methods specified in therelationship definition and computation templates 110. Indexing of thesource data item by an integer is helpful in that it eliminates the needto traverse the entire navigation tree structure to retrieve the dataitem's non-display attributes, and it also avoids the necessity ofinserting all the attributes of every data item into the interface sothat any user request would have all appropriate attribute valuesaccessible to pass to the query handler. Note that data items, ifdesired, can be linked to large binary files, such as computer-aideddesign (CAD) files for automobile and airplane parts, which can bespecified, for example, in the binary links list depicted in attributes516. In view of the discussion of FIG. 5, it will be appreciated thatthe navigation tree structure can include a plurality of nodes 508 thatare configured to enable presentation to the user of a depth-firsttraversal of the navigation tree structure 500 and that the structurecan further include the node data array 506 wherein pertinent data,typically including at least an entity type and at least one attribute,is stored for each of the plurality of nodes 508. Again, as noted, eachof the nodes 508 in the navigation tree portion 504 of the navigationtree structure 500 can include an attribute designated as a displayattribute 502, a relationship 510, a node number 514, and a next level512, and the attributes can be used as parameters with the relationshiptemplates 110. Note that an entity can be envisioned as an object thatone desires to present to a user to interface with to allow navigationthrough the system.

Reference should now be had to FIG. 6, which depicts a generalizednavigation tree 604. The navigation tree 504 of FIG. 5 has been expandedfor depiction in FIG. 6, and like elements have received the samereference character incremented by 100. Elements 602, 608, 610, 612, 614are similar to elements 502, 508, 510, 512, 514 in FIG. 5 and are notdiscussed again. In order to complement the flexibility of therelationship definition and computation templates 110 in specifyingmulti-dimensional relationships (where an entity is linked to otherentities through several relationships), as well as the expressive powerof the system in terms of general application, the navigation treeportion 504 in FIG. 5 has been extended as a generalized navigation treedata structure 604 in FIG. 6. By way of example, FIG. 6 illustrates howthe “exterior” entity can be associated to other entities through boththe “coordinator” and “subassembly-parts” relationships. The generalizednavigation tree data structure is thus built up of generic nodes 620that keep track of all pertinent relationships navigated by the user fora given entity, in this case, “exterior”.

Attention should now be given to FIG. 7 which diagrams amulti-dimensional navigation tree structure 704 and a generalized graph722. Again, to the extent that there is commonality between itemsdepicted in FIG. 7 and those depicted in FIG. 5 and FIG. 6, referencecharacters in FIG. 7 have been incremented by 100 with respect to FIG. 6and by 200 with respect to FIG. 5. FIG. 7 also depicts a user drill-downnavigation path 724 with inactive zones 726. Suppose that users wereallowed to navigate along exactly one relationship for each entity. Insuch a case, the navigation tree could resemble, for example, just theunhatched structure 727 (corresponding, e.g., to the “subassembly”relationship), just the single-hatched structure 728 (corresponding,e.g., to the “management” relationship), or just the double-hatchedstructure 729 (corresponding, e.g., to some other relationship).However, because the present invention permits the definition andcomputation of multiple relationships per entity, one should permitnodes to be traversed along multiple relationships, leading to themulti-dimensional navigation tree structure 704. The top dotted box in704, containing an unhatched circle and a single-hatched circle, canrepresent, for example, two relationships (e.g., subassembly,management) applying to the “exterior” entity. While the structure issomewhat complex in appearance, we can use the generalized navigationtree structure 704 and its generic nodes (represented by the circles andnot separately numbered) to represent the multi-dimensional navigationtree structure as an easily extensible and cleanly organized flat (ortwo-dimensional) generic graph 722. In this form, then, one can keeptrack of an n-dimensional relationship graph in a data tree structurethat is easily scalable for efficient navigation and traversal. Now,both the relationship definition/computation scripts and the underlyingin-memory data structures allow for a high level of flexibility in termsof describing relationships, in particular, as a general graph.

Note that, depending on the viewing scheme, several dimensions of anavigation structure are potentially lying “inactive” in the background,unviewed by the user at the current moment. Thus, entire dimensions canbe adaptively persisted to storage (for example, a disk or other type ofmemory, often a non-volatile form of memory) and brought back to thememory as the navigation tree structure requires with respect to theuser's view of the navigation tree structure. If the user has drilleddown a certain path in the navigation tree structure, represented byheavy line 730, leaving the rest of the navigation tree structureunexpanded, a significant fraction of the relationship graph will beinactive, as indicated by regions 726. Given the graph representation ofthe present invention, these “inactive” zones can be efficientlylocated, and the granularities at which these zones can be persisted tomemory (again, typically a disk or other type of non-volatile memory,but any appropriate form of memory can be used) extend very naturallyfrom the size and location of the sub-graphs or sub-trees that make upthe zones 726.

In view of the foregoing description of FIG. 7, it will be appreciatedthat the platform of the present invention can, where desired, have anavigation tree structure that includes both active and inactiveregions; the inactive regions, as noted, are depicted as item 726 inFIG. 7, while the remaining nodes in item 724 constitute the activeregion. Further, the inactive region 726 of the navigation treestructure can be persisted to the relationship and navigation datarepository 108, while active regions of the navigation tree structurecan be formatted in a manner adapted to be output from the platform 100in a readily accessible fashion. For example, the formatting can beconducted by the interface module 122 for display to a user employing anexternal browser 106.

Reference should now be had to FIG. 8, which depicts a flow chart for amethod, in accordance with the present invention, of constructing anavigation tree structure (which can be of the type discussed above) formanaging a plurality of distributed heterogeneous data sources (such asthe data sources 102). The method 800 includes the step 802 of obtainingpre-existing definitional data. The pre-existing definitional data canbe, for example, a result stored in a back-end, such as one of the datasources 102, as depicted at block 804; a result from persistent storage,such as central data model 104 or relationship and navigation datarepository 108, as depicted at block 806; or a subgraph from persistentstorage, as depicted at block 808. As depicted at block 810, the methodfurther includes the step of instantiating a plurality of in-memorynodes for the pre-existing definitional data (the in-memory nodes canoptionally be instantiated in an iterative manner); and as depicted atblock 812, initializing a relationship attribute and/or an entityattribute for each of the nodes. The relationship attributes can definerelationships among data in the same data source, and unlike previoussystems, can also define relationships among data in different datasources. Method 800 can further include the step of forming an updatednavigation tree structure from the aforementioned nodes having the givenattributes, per block 814. In the case when the pre-existingdefinitional data is a sub-graph, the instantiating, initializing, andforming steps can be repeated for each level of nodes, as depicted atblock 824. Note that a subgraph, like any graph, can be thought of as ahierarchy of any arbitrary number of levels of nodes.

As indicated at blocks 816, 818, 820, method 800 can optionally includethe additional step of determining a memory constraint associated with afirst memory, comparing the updated navigation tree structure to thememory constraint, and then, if the updated navigation tree structureexceeds the memory constraint, performing additional steps. Theadditional steps to be performed in such case include determining firstportions of the navigation tree structure that are inactive and secondportions of the navigation tree structure that are active, storing atleast the first portions of the navigation tree structure to a secondmemory, and retaining in the first memory the second portions of thenavigation tree structure (block 822). Examples of active and inactiveportions have been discussed above with respect to FIG. 7. By way ofexample and not limitation, the first memory can typically include aRAM, while the second memory can typically include a disk or otherpersistent storage embodied in relationship and navigation datarepository 108, and particularly in navigation assembly and structurerepository 112. The memory constraint can be associated with, forexample, the number of users or the size of the navigation tree datastructure, as well as the available memory resources.

System and Article of Manufacture Details

As is known in the art, the methods and apparatus discussed herein maybe distributed as an article of manufacture that itself comprises acomputer readable medium having computer readable code means embodiedthereon. The computer readable program code means is operable, inconjunction with a computer system, to carry out all or some of thesteps to perform the methods or create the apparatuses discussed herein.The computer readable medium may be a recordable medium (e.g., floppydisks, hard drives, compact disks, or memory cards) or may be atransmission medium (e.g., a network comprising fiber-optics, theworld-wide web, cables, or a wireless channel using time-divisionmultiple access, code-division multiple access, or other radio-frequencychannel). Any medium known or developed that can store informationsuitable for use with a computer system may be used. Thecomputer-readable code means is any mechanism for allowing a computer toread instructions and data, such as magnetic variations on a magneticmedia or height variations on the surface of a compact disk.

The computer systems and servers described herein each contain a memorythat will configure associated processors to implement the methods,steps, and functions disclosed herein. The memories could be distributedor local and the processors could be distributed or singular. Thememories could be implemented as an electrical, magnetic or opticalmemory, or any combination of these or other types of storage devices.Moreover, the term “memory” should be construed broadly enough toencompass any information able to be read from or written to an addressin the addressable space accessed by an associated processor. With thisdefinition, information on a network is still within a memory becausethe associated processor can retrieve the information from the network.

It is to be understood that the embodiments and variations shown anddescribed herein are merely illustrative of the principles of thisinvention and that various modifications may be implemented by thoseskilled in the art without departing from the scope and spirit of theinvention.

1. A system integration apparatus for managing, in response to a querycommand, a plurality of distributed heterogeneous data sources and acentral data model configured to contain at least one of entities andattributes that are at least one of mapped and translated from the datasources, and for determining a desired query result corresponding to thecommand, each of the data sources having data that can flow into saidapparatus, said apparatus comprising: a relationship and navigation datarepository configured to contain relationship templates definingrelationships among the data from the data sources, said relationshiptemplates defining said relationships among both data within individualones of the sources and data in different ones of the sources, saidrelationship and navigation data repository being configured tointerface with the central data model; and an integration engine coupledto said relationship and navigation data repository, said integrationengine being configured to: receive the query command, determine whichof the plurality of data sources and which of said templates the querycommand applies to, so as to identify pertinent data sources andpertinent templates; and calculate the desired query result, responsiveto the query command, based on the pertinent data sources and saidpertinent templates.
 2. The apparatus of claim 1, further comprising anapplication server, said application server in turn comprising: saidintegration engine; and an interface module that is coupled to saidintegration engine and through which said integration engine receivesthe query command.
 3. The apparatus of claim 2, wherein: saidintegration engine calculates the result formed as a navigation treestructure having a plurality of source data items; and said interfacemodule is configured to format at least part of said navigation treestructure in a manner adapted to be output from said apparatus in areadily accessible fashion.
 4. The apparatus of claim 3, wherein saidinterface module formats at least said part of said navigation treestructure into a query interface, said query interface comprising: datarepresentative of a first portion for specification of a desired one ofsaid source data items from said navigation tree structure; and datarepresentative of a second portion for specification of a pertinent oneof said relationship templates.
 5. The apparatus of claim 4, whereinsaid query interface further comprises data representative of a thirdportion for specification of a desired depth of navigation into saidnavigation tree structure.
 6. The apparatus of claim 3, wherein saidrelationship and navigation data repository is further configured tostore said navigation tree structure.
 7. The apparatus of claim 3,wherein: said navigation tree structure in turn comprises a plurality ofnodes configured to enable presentation to the user of a depth-firsttraversal of said navigation tree structure; and said navigation treestructure further comprises a node data array with pertinent data,including at least an entity type and at least one attribute, for eachof said plurality of nodes.
 8. The apparatus of claim 7, wherein: eachof said nodes in said navigation tree structure includes at least: oneof said attributes designated as a display attribute, a relationship, anode number, and a next level; and said attributes in said navigationtree structure are used as parameters with said relationship templates.9. The apparatus of claim 8, wherein: said navigation tree structureincludes both active and inactive regions; and said inactive regions ofsaid navigation tree structure are persisted to said relationship andnavigation data repository while active regions of said navigation treestructure comprise said part of said navigation tree structure that isformatted in a manner adapted to be output from said apparatus in areadily accessible fashion.
 10. The apparatus of claim 1, wherein saidrelationship and navigation data repository further comprises: aplurality of relationship definitions defining a plurality ofrelationships, each of said relationship definitions identifying asource entity, an associated destination list of pertinent ones of thedata sources, and at least one computation definition reference; and aplurality of computation definitions defining how to computecorresponding ones of said relationships responsive to said at least onecomputation definition reference of a corresponding one of saidrelationship definitions.
 11. A method of constructing a navigation treestructure for managing a plurality of distributed heterogeneous datasources, said method comprising the steps of: obtaining pre-existingdefinitional data; instantiating a plurality of in-memory nodes for saidpre-existing definitional data; initializing at least a relationshipattribute and an entity attribute for each of said nodes, whereby aplurality of relationship attributes are initialized, at least some ofsaid relationship attributes defining relationships among data indifferent ones of the data sources; and forming an updated navigationtree structure from said nodes having said attributes.
 12. The method ofclaim 11, wherein: said pre-existing definitional data is one of aresult stored in one of the data sources and a result stored in arelationship and navigation data repository, and said in-memory nodesare instantiated iteratively.
 13. The method of claim 12, furthercomprising the additional steps of: determining a memory constraintassociated with a first memory; comparing said updated navigation treestructure to said memory constraint; and if said updated navigation treestructure exceeds said memory constraint: determining first portions ofsaid navigation tree structure that are inactive and second portions ofsaid navigation tree structure that are active; storing at least saidfirst portions of said navigation tree structure to a second memory; andretaining in said first memory said second portions of said navigationtree structure.
 14. The method of claim 11, wherein: said pre-existingdefinitional data is a sub-graph stored in a relationship and navigationdata repository and having a plurality of node levels, and saidin-memory nodes are instantiated iteratively; said method furthercomprising the additional step of: repeating said instantiating,initializing, and forming steps for each level of said plurality of nodelevels.
 15. The method of claim 14, further comprising the additionalsteps of: determining a memory constraint associated with a firstmemory; comparing said updated navigation tree structure to said memoryconstraint; and if said updated navigation tree structure exceeds saidmemory constraint: determining first portions of said navigation treestructure that are inactive and second portions of said navigation treestructure that are active; storing at least said first portions of saidnavigation tree structure to a second memory; and retaining in saidfirst memory said second portions of said navigation tree structure. 16.An article of manufacture for constructing a navigation tree structurefor managing a plurality of distributed heterogeneous data sources,comprising a machine readable medium containing one or more programswhich when executed implement the steps of: obtaining pre-existingdefinitional data; instantiating a plurality of in-memory nodes for saidpre-existing definitional data; initializing at least a relationshipattribute and an entity attribute for each of said nodes, whereby aplurality of relationship attributes are initialized, at least some ofsaid relationship attributes defining relationships among data indifferent ones of the data sources; and forming an updated navigationtree structure from said nodes having said attributes.
 17. The articleof manufacture of claim 16, wherein: said pre-existing definitional datais one of a result stored in one of the data sources and a result storedin a relationship and navigation data repository, and said in-memorynodes are instantiated iteratively.
 18. The article of manufacture ofclaim 17, further comprising the additional steps of: determining amemory constraint associated with a first memory; comparing said updatednavigation tree structure to said memory constraint; and if said updatednavigation tree structure exceeds said memory constraint: determiningfirst portions of said navigation tree structure that are inactive andsecond portions of said navigation tree structure that are active;storing at least said first portions of said navigation tree structureto a second memory; and retaining in said first memory said secondportions of said navigation tree structure.
 19. The article ofmanufacture of claim 16, wherein: said pre-existing definitional data isa sub-graph stored in a relationship and navigation data repository andhaving a plurality of node levels, and said in-memory nodes areinstantiated iteratively; further comprising the additional step of:repeating said instantiating, initializing, and forming steps for eachlevel of said plurality of node levels.
 20. The article of manufactureof claim 19, further comprising the additional steps of: determining amemory constraint associated with a first memory; comparing said updatednavigation tree structure to said memory constraint; and if said updatednavigation tree structure exceeds said memory constraint: determiningfirst portions of said navigation tree structure that are inactive andsecond portions of said navigation tree structure that are active;storing at least said first portions of said navigation tree structureto a second memory; and retaining in said first memory said secondportions of said navigation tree structure.
 21. A method of processing aquery command associated with a plurality of distributed heterogeneousdata sources, said method comprising the steps of: obtaining the querycommand; responsive to the query command, performing one of creating anavigation session and loading a pre-existing navigation session;passing the query command and the corresponding one of the newnavigation session and the pre-existing navigation session to anintegration engine; identifying and interpreting, via said integrationengine, pertinent definition and computation templates associated withthe query command, at least some of said templates definingrelationships among data in different ones of the data sources; andcomputing and retrieving a desired query result, based at least in parton at least one of said definition and computation templates, frompertinent ones of the distributed heterogeneous data sources.
 22. Themethod of claim 21, further comprising the additional step of outputtingat least part of said desired query result formatted as a navigationtree structure.
 23. The method of claim 21, further comprising theadditional step of writing said desired query result to persistentstorage, said desired result being formatted as a navigation treestructure.
 24. An article of manufacture for processing a usernavigation request associated with a plurality of distributedheterogeneous data sources, comprising a machine readable mediumcontaining one or more programs which when executed implement the stepsof: obtaining the query command; responsive to the query command,performing one of creating a navigation session and loading apre-existing navigation session; passing the query command and thecorresponding one of the new navigation session and the pre-existingnavigation session to an integration engine; identifying andinterpreting, via said integration engine, pertinent definition andcomputation templates associated with the query command, at least someof said templates defining relationships among data in different ones ofthe data sources; and computing and retrieving a desired query result,based at least in part on at least one of said definition andcomputation templates, from pertinent ones of the distributedheterogeneous data sources.
 25. The article of manufacture of claim 24,further comprising the additional step of outputting at least part ofsaid desired query result formatted as a navigation tree structure. 26.The article of manufacture of claim 24, further comprising theadditional step of writing said desired query result to persistentstorage, said desired result being formatted as a navigation treestructure.
 27. A system for constructing a navigation tree structure formanaging a plurality of distributed heterogeneous data sources, saidsystem comprising: a relationship and navigation data repository; and anintegration engine coupled to said relationship and navigation datarepository, said integration engine being configured to: obtainpre-existing definitional data from at least one of: said relationshipand navigation data repository, and at least one of the distributed datasources; instantiate a plurality of in-memory nodes for saiddefinitional data; initialize at least a relationship attribute and anentity attribute for each of said nodes, whereby a plurality ofrelationship attributes are initialized, at least some of saidrelationship attributes defining relationships among data in differentones of the data sources; and form an updated navigation tree structurefrom said nodes having said attributes.
 28. The system of claim 27,wherein: said pre-existing definitional data is one of a result storedin one of the data sources and a result stored in said relationship andnavigation data repository, and said in-memory nodes are instantiatediteratively by said integration engine.
 29. The system of claim 28,wherein said integration engine is further configured to: determine amemory constraint associated with a first memory; compare said updatednavigation tree structure to said memory constraint; and if said updatednavigation tree structure exceeds said memory constraint: determinefirst portions of said navigation tree structure that are inactive andsecond portions of said navigation tree structure that are active; storeat least said first portions of said navigation tree structure to asecond memory; and retain in said first memory said second portions ofsaid navigation tree structure.
 30. The system of claim 27, wherein:said pre-existing result is a sub-graph stored in said relationship andnavigation data repository, said sub-graph having a plurality of nodelevels; said in-memory nodes are instantiated iteratively by saidintegration engine; and said integration engine is configured torepeatedly instantiate, initialize, and form for each level of saidplurality of node levels.
 31. The system of claim 30, wherein saidintegration engine is further configured to: determine a memoryconstraint associated with a first memory; compare said updatednavigation tree structure to said memory constraint; and if said updatednavigation tree structure exceeds said memory constraint: determinefirst portions of said navigation tree structure that are inactive andsecond portions of said navigation tree structure that are active; storeat least said first portions of said navigation tree structure to asecond memory; and retain in said first memory said second portions ofsaid navigation tree structure.